The present invention relates to a stereophotogrammetry device, preferably portable and a method to acquire, reconstruct in 3D and measure characteristics and changes in 3-Dimensions of the head and body surfaces.
For Plastic Surgery, 3D reconstruction is necessary to reconstruct the surfaces of body parts in order to measure the geometry of subjects, analyze their shapes, simulate surgical procedures and also to collect body surface information over time in order to compare and measure geometric variations.
Plastic surgeons are particularly interested in the 3D surface of the face for surgical procedures such as rhinoplasty and various lift procedures. Other areas of interest include the 3D surface of the torso, breast surgery and breast implant selection as well as other parts of the body such as buttocks, hips, neck, sternum region, hands and various other areas. The needs in these areas correspond to large variations in the field of view given that the surface of a complete torso is much larger than of a face.
Stereophotogrammetry consists in gathering the images of a subject from at least two view with a calibrated camera whose optics are perfectly modeled. When two such images are acquired simultaneously, one generally refers to it as a stereo-pair and by finding corresponding points between the two images of the stereo-pair using cross-correlation types of algorithms, one can reconstruct via triangulation a dense representation in 3-Dimensions of the surface of the observed object.
Calibration of the optics is one of the key steps necessary for accurate 3-Dimensional reconstruction. The geometric accuracy necessary to build the optics is such that in practice it is not possible to have them with mobile parts, which means that stereophotogrammetry devices used to reconstruct anatomical surfaces have a fixed focal plane. Along with the aperture of the optics, this defines a fixed depth of field spaced around the focal plane and a fixed field of view.
Thus, for 3D reconstruction of the face, a specialist will use a stereophotogrammetry camera with a focus distance corresponding to a field of view of approximately an A4 surface format that is 21 cm times 29.7 cm. For breast 3D reconstruction, a specialist would use a stereophotogrammetry camera with a focus distance corresponding to a field of view of approximately an A3 surface format that is 29.7 cm times 42 cm.
In practice, a specialist is using two distinct stereophotogrammetry cameras for acquiring on one side, the 3D representation of faces and on the other, the 3D representations of the breast.
In general, it is necessary to add to the stereophotogrammetry system some means to position the subject at a distance corresponding to the focus plane. One way to achieve such a fixed distance positioning is to use a portable stereophotogrammetry system such as the one described in “MAVIS: a non-invasive instrument to measure area and volume of wounds. Measurement of Area and Volume Instrument System”, by Plassmann P, Jones T D, Med. End. Phys. 1998; 20(5):332-8. Such stereophotogrammetry system is equipped with a pair of light projectors—also called light beamers or pointers—converging at a fixed distance which correspond to the focus plane of the stereo camera.
In the case of a curved anatomical surface such as a face or breast, some important parts are not visible simultaneously from both optics of the stereo camera. This can be due to the curvature of the subject and is independent from the width of the field of view of the device. As a consequence, some parts of the subject cannot be reconstructed in 3D from a single stereo-pair regardless of the width of the field of view.
For such reason, static multi-heads 3-Dimensional systems have been developed in order to take simultaneously several stereo-pairs of the same subject, in order to cover the “dead angles”. By calibrating the exact position of these different camera heads thanks to calibration targets, it is then possible to “stitch” the different 3D surfaces obtained in order to generate a single 3D surface representation without hidden parts. This has been done for the head as well as for breast and torso 3D reconstructions.
This technique which is delicate to implement is called “stitching”. Stitching has been used originally to piece together different 2D images in order to produce a 2D panoramic scenery picture. It has been extended to 3D surfaces and associated image textures in order to build single 3D representations of a subject's surface from several individual surface pieces. Typically, these static multi-head systems are using three to four heads depending upon the applications.
The “LifeViz II” as described in “Exhibition Watch Report—in Cosmetics 2013” 2013, XP055139703, Paris, pages 8 and 9 which the system's design is credited the author of the present disclosure. The system is composed of a portable stereophotogrammetry camera with light beamers and computation means which enable the stitching of the 3D surfaces obtained from multiple stereo-pairs in order to reconstruct in 3-Dimensions a comprehensive surface of a face or breast.
The difficulty solved by this system is to be able to stitch different views at different time points and according to different viewpoints corresponding to unknown angles. This system is in contrast to fixed multi-head systems for which the different viewpoints are perfectly known geometrically via a-priori calibration.
In order to develop the LifeViz II, it was necessary to develop innovative 3D surface matching algorithms enabling to automatically retrieve each position of the camera when it was used to take each of the views. Other innovative algorithms were necessary to fuse the texture maps acquired with different light conditions as well as surfaces which may have slightly deformed between each view given the subject may have moved or deformed between snapshots.
The LifeViz II is considered the state of the art relative to portable stereophotogrammetry systems. It is limited by the fact that it needs to be built and optimized for one single fixed distance for image acquisition. Hence, there are stereophotogrammetry devices optimized for face 3D reconstruction and for which the field of view is approximately equivalent to an A4 surface and other models optimized for the 3D reconstruction of the torso and for which the field of view is approximately equivalent to an A3 surface area.
Indeed, with a field of view close to A4 surface format, it is possible to cover a face with 3 or 4 views with a portable system. With a portable system and a field of view restricted to A4, and because it is necessary to get some overlap between the different surfaces in order to match and stitch them, using such a system for the torso would necessitate in practice too many A4 views. Furthermore, it would be quite difficult to manually position the camera in space as well as keep the subject still for the numerous views required with an A4-type of device.
By using a system whose field of view corresponds to an A3 format, and in order to avoid the dead angles, one can cover the surface of a torso with three to five views and with relatively simple instructions for a subject's positioning.
With a system whose field of view corresponds to an A3 format, one can also acquire the stereo-pairs necessary to reconstruct a face but with a much reduced resolution given the surface of the image used to represent the face is very restricted and many pixels are unused.
To conclude, portable stereophotogrammetry devices currently developed are including a single, nominal distance for picture taking which is optimized either for the face or for breast, but not optimized for both applications at the same time. A specialist wanting to use a portable system for both the head and torso simultaneously would use a system designed for torso but will lose accuracy when the face surface 3D reconstruction will be needed.